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ABSTRACT 

The process of developing indicators, measures, and 
standards of program quality can be considered a tiered, four-step 
process consisting of the following: (1) identification of topic 
areas important to program quality; (2) establishing the indicators 
that reflect quality for each area; (3) determining measures of the 
indicators; and (4) setting a numeric criterion or standard for the 
measure. Topic areas include program context, program process and 
content, and program outcomes. Quality indicators are variables that 
reflect effective and efficient performance. Performance measures are 
the data used to determine the quantitative level of performance. 
Performance standards are measures with a specific numeric criterion 
or level of performance tied to them. Indicators and performance 
standards can be used to assess the success of a program and to hold 
programs accountable for performance. Indicators can be defined using 
the focus group method. (Two tables provide examples of indicators, 
measures, and standards and a framework for development of quality 
indicators.) (KC) 
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QUALITY INDICATORS, MEASURES AND PERFORMANCE STANDARDS: 
OVERVIEW AND DEFINITIONS 



The process of developing indicators, measures and standards of program quality can be 
considered a tiered, four-step process consisting of (1) identification of topic areas important to 
program quality, (2) establishing the indicators that reflect quality for each area, (3) determining 
measures of the indicators and (4) setting a numeric criterion or standard for the measure. Each 
step is discussed below. Since the terms "indicators," "performance standards" and "measures" are 
often used interchangeably, definitions are provided to clarify the terms for our discussions. 

• Topics. The Department of Education has identified three general topic areas to 
develop quality indicators for ABE programs: program context, program process 
and content, and program outcomes. Within these three general areas, Pelavin 
Associates has identified more specific topics areas based on a review of state 
monitoring instruments, evaluation studies and funding protocols. The general 
areas are defined as follows. 

Program context — the setting in which the program operates including 
characteristics of participants, organization of the delivery system and the needs 
and demographics of the community. 

Program process and content - components of the program that define how it 
operates, such as program planning, recruitment of students, intake and 
assessment, staff characteristics, curriculum and instructional content, materials and 
equipment used, assessment of student progress, evaluation and follow-up. 

Program outcomes — impact of the program on students, such as learning gains and 
goal attainment. 

• Indicators. Once topics have been agreed on, indicators of quality are established 
within each area. A quality indicator is a variable that reflects effective and 
efficient performance. For example, retention of students in a program may be a 
indicator of quality. In adult education, it is difficult to establish indicators in 
many areas, since there is a lack of agreement and empirical research on what 
constitutes effective and efficient practice. 

• Measures. A performance measure is the data used to determine the quantitative 
level of performance. The measure is the operational definition of the indicator. 
For example, retention may be measured by the number of hours students remain 
in the program. 

• Performance standards. A performance standard is a measure with a specific 
numeric criterion or level of performance tied to it. A retention performance 
standard would specify a minimum number of hours of instruction per student, for 
example. The standard defines a level of acceptable performance on the measure. 
Standards may be established for a single point in time or to measure increases in 
performance over time. Among Federal programs, the Job Training Partnership 
Act (JTPA) programs have the most well known system of performance standards. 



The following exhibit provides examples of indicators, measures and standards. 

Uses of Indicators and Performance Standards 

Indicators provide clear and unambiguous methods for assessing the success of a program 
in meeting goals. They promote program improvement by pointing to areas of excellence and 
weakness. Ideally, indicators will help all users of adult education services and those responsible 
for administering them. Policy makers could use the information from indicators to inform 
decisions. Administrators could assess the effectiveness of programs to improve them to better 
meet the needs of students and the community. Students, employers and other users of adult 
education services could use indicators to compare the effectiveness of different programs. The 
set of papers on indicators you received discusses some of the implications of using indicators for 
different types of providers and programs. 

Performance standards serve the same functions as indicators but also are used to hold 
programs accountable for performance by adding a system of incentives for meeting the standards 
or sanctions for failing to meet them. There are potential dangers, however, to using 
performance standards, as they may produce unintended effects on program design and 
participant characteristics. Many of these issues are discussed in the Quality Indicators for Adult 
Education Programs: Lessons Learned from Other Programs, prepared by Pelavin Associates. 

Goals of Focus Group Meetings 

With these dual uses in mind, it is our goal to develop a list of indicators of program 
quality. Within the three major topic areas specified by OVAE, Pelavin Associates has identified 
a preliminary list of additional topics and indicators, as described in the report Synthesis of State 
Quality Indicators for Adult Education Programs. The attached exhibit presents these topics using 
the framework discussed here. At four focus group meetings, we will develop a list of quality 
indicators by: (1) discussing and refining the topics identified, adding or deleting topics as 
necessary; (2) discussing the specific indicators and reaching consensus on them for each topic; 
and (3) prioritizing the indicators to develop a manageable number. We will also discuss 
measurement of some indicators as needed to clarify or refine their meaning. However, we will 
not deal with establishing performance standards as part of the meeting. 

We do not want a laundry list of indicators covering every possible dimension of adult 
education programs. The final list of indicators should: 

• Be a manageable number so as not to overburden state and local programs with 
data collection; 

• Be measurable without excessive burden; 

• Represent the essential elements of programs that define quality; 

• Accommodate the diverse nature of the adult education delivery system; and 

• Be sensitive to the diverse needs of learners. 
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